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1 Introduction 



In this paper we investigate the optimal stochastic control problem for stochastic differ- 
ential equations (SDEs) reflected on a given domain. The cost functional is specified by 
controlled generalized backward stochastic differential equations (GBSDEs) which de- 
pend on the reflections in the forward SDEs. The associated Hamilton-Jacobi-Bellman 

(HJB) equation turns out to have a nonlinear Neumann boundary condition. To be 
more detail, we aim to give the stochastic representations for the solutions of the fol- 
lowing HJB equations with a nonlinear Neumann boundary condition: 



a 



W{t, x) + H{t, X, W, DW, D^W) = 0, {t, x) G [0, T) x D, 



^^Wit,x)+g{t,x,W{t,x)) =0, 
WiT,x) = $(x), 

d a 



0<t<T,xedD; 

X e D, 



(1.1) 



where at a point x G dD, ^ = Yli=i '^^{^)'^-> ^'^^ Hamiltonian 



H{t,x,y,p,X) = sup <^ -tr{aa^{t,x,u)X) +p.b{t,x,u) + f{t,x,y,p.a,u) 

where t e [0,T], x G M", y G M, p G M'^, and X G S"^. Here the functions b, a, f, and $ 
are supposed to satisfy (H3.1) and (H3.2), respectively. D is an open connected 
bounded convex subset of W^, more details in Section 3. For this, we consider the 
following stochastic system: For an admissible control u(-) G U, the corresponding 
state process starting from x G .D at the initial time t, is governed by the following 
reflected SDE: 



X + 



, X^'^'", Ur)dr + J' a{r, X^'^'", Ur)dBr 



h{r, 



V0(x*^"'")(iA7'^'", s G [t,r] 



(1.2) 



Ki''^'" = j I^xt'^;u^Q^ydKl'^''', ii:*'^'" is increasing, 



and the associated GBSDE is the following: 



f{s,Xl'--'\Y,'^--^^,Zl'--'\us)ds 
+g{s, X*'^'", y,*'^'") di^*'^''' - Z*'^'" dB„ 



:i.3) 



Under standard assumptions (H3.1) and (H3.2), they have unique solutions (X*'^'", X*'^'") 
and (y*'^;", Z*'^'"), respectively. For any u{-) G Ut,T, the cost functional is defined by 



J{t,x;u) := y/'""'", {t,x) G [0,r] x D. 
The value function of our stochastic control problem is as follows: 
W{t, x) := csssup J{t, x; u), (t, x) G [0, T] x D. 
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(1.4) 



(1.5) 



We prove that the value function is a viscosity sohition of HJB equation (1.1). 

The general formulation of backward stochastic differential equations (BSDEs) was 
first introduced by Pardoux and Peng [14j in 1990, who proved existence and uniqueness 
of adapted solutions for these equations under suitable assumptions on both the coeffi- 
cients and the terminal condition. Since then they have been studied by many authors 
and have various applications, for example, in stochastic control, finance and partial 
differential equations (PDEs). The reader is referred to, among others, EL Karoui, 
Peng and Quenez [9] , Darling and Pardoux [6] , Pardoux and Peng [15] , Peng [H] , [H] , 
Hu |10) . and Delbaen and Tang 0. Stochastic differential equations with reflecting 
boundary conditions have been studied by many authors, see Lions [11], Lions and 
Sznitman [12], Menaldi |13j . Pardoux and Williams [16], Saisho |20j . among others. 
Note that in the work of Pardoux and Zhang [17], a new class of BSDEs, which is 
driven by, in addition to a Brownian motion, a continuous increasing process, is stud- 
ied and used to give a probabilistic formula for the solution of a system of parabolic or 
elliptic semi-linear partial differential equation with a nonlinear Neumann boundary 
condition. There are many related works, and the reader is referred to Boufoussia and 
Van Casterenb [2], Day [7], etc. 

The BSDE method developed by Peng [18] and [19] for the dynamic programming 
of optimal stochastic control, is extended into our context. We shall also adapt the ar- 
guments of Buckdahn and Li [4j to show that our value function W (see (j3.7p ) , which is 
introduced as the essential supremum over a class of random variables, is deterministic 
(see Proposition 3.1). Such an effort is not trivial since the cost is also driven by the 
increasing process to incorporate the reflection of the system state on the boundary 
of the given domain. It is crucial for us to show that it is the continuous viscosity 
solution to the associated HJB equation subject to a nonlinear Neumann boundary 
condition (Theorem 4.1). It allows us to prove the dynamic programming principle 
(DPP in short, see Theorem 3.1) in a straight forward way by adapting to GBSDEs 
the method of stochastic backward semigroups introduced by Peng [18j. However, the 
proof becomes more technical due to the Neumann boundary condition. We have to 
improve the estimate on GBSDE of Pardoux and Zhang [17]. We could prove that, 
in the Markovian framework and under standard assumptions, the dependence of the 
solution on the random initial value of the driving SDE is locally Lipschitz (Proposi- 
tion 5.2), and the increasing process has a new important estimate (Proposition 5.3). 
Furthermore, our proof of Theorem 4.1, which states that the value function W is 
a continuous viscosity solution of the associated HJB equations with the Neumann 
boundary condition, differs heavily from those counterpart of either Buckdahn and 
Li [3] or Peng [18]. For the detailed complication, the reader is referred to among 
others Lemmas 4.2 and 4.3 and the constructions of BSDEs KTO\\ . (f4T2]) . ([423]) and 
(j4.24p . In our context, the coefficients of the problem are not necessarily continuous 
in the control variable u, and the control u may take values in a possibly noncompact 
space U. 

Our paper is organized as follows. In Section 2, we recall some elements of the 
theory of BSDEs and GBSDEs. In Section 3, we introduce the setting of our stochastic 
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control problem, the nonlinear Neumann boundary condition, and the value function 
W. We prove that W is deterministic and satisfies the DPP, which implies that W 
is continuous. In Section 4, we prove that is a viscosity solution of the associated 
HJB equation with a nonlinear Neumann boundary condition. For the sake of readers, 
some necessary basic properties of GBSDEs associated with forward reflected SDEs, 
are documented in the appendix (Section 5), where Propositions 5.2 and 5.3 contain 
new results on GBSDEs. 

2 Preliminaries 

Consider the Wiener space {Q,J-,P), where 17 is the set of continuous functions from 
[0, T] to R'^ starting from (il = Co([0, T]; M'^)), T the completed Borel ir-algebra 
over il, and P the Wiener measure. Let B be the canonical process: Bs{uj) = Ug, s € 
[0,T], a; € il. By F = {Ts, < s < T} we denote the natural filtration generated by 
{Bs}o<s<T and augmented by all P-null sets, i.e., 

IS = a{Br,r < sjV M, sG[0,r], 

where Af is the set of all P-null subsets, and T > a fixed real time horizon. For any 
n > 1, \z\ denotes the Euclidean norm of z G R". Introduce the following two spaces 
of processes: 5^(0, r;R) is the collection of {ipt)o<t<T which is a real- valued adapted 
cadlag process such that E[ sup iV-tP] < +0O, and n^{0,T;R'') is the collection of 

0<t<T 

{'ipt)o<t<T which is an M^-valued progressively measurable process such that || ip ||2= 
E[f^ \il;t\'^dt] < +00. 

We make the following assumptions on g throughout the paper. 

(Al) The function 5 : x [0, T] x M x M'^' ^ M has the property that {g{t, y, z))te[o,T] 
is progressively measurable for each (y, z) in M x M*^. There exists a constant C > 
such that, P-a.s., for ah t e [0,r], yi,y2 G M, 2:1,22 G M'^, 

\g{t,yi,zi) -g{t,y2,Z2)\ < C{\yi -2/2I + \zi - Z2\). 

(A2) 5(-,0,0) £nHO,T;R). 

The following result is due to Pardoux and Peng [14] on BSDEs. 
Lemma 2.1. Under assumptions (Al) and (A2), the BSDE 

yt = C+ [ g{s,ys,zs)ds- I ZsdBs, 0<t<T, (2.1) 
Jt Jt 

has a unique adapted solution {y,z) G 5^(0, T;M) x 'H'^{0,T;R'^) for any random vari- 
able C G L'^{0,Tt, P). 

We need the following comparison theorem on BSDEs, which can be found in El 
Karoui, Peng, and Quenez [9]. 

Lemma 2.2. (Comparison Theorem) Let gi and 52 satisfy (Al) and (A 2) and ^1,^2 £ 
L'^{n,TT,P). Denote the solution of BSDE by {y^,z'^) for the data {^,g) = 

(Cii^i) o,i^d by {y'^,z^) for the data (Cfi') = {£,2t92)- Then we have 
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(i) (Monotonicity) If > ^2 o^iT-d gi > 92, a.s., then yj > y|, a.s. for all t G [0, T]. 

(ii) (Strict Monotonicity) If P{^i > ^2) > m addition to (i), then P{yl > y^} > 
for t € [0,T], and in particular, y^ > y^. 

Lemma 2.3. Suppose that the randam field g : Q x [0,T] xRxR'^ — > M satisfies (Al) 
and (A2), and the two random variables ^1,^2 £ L'^{^,J'T,P)- Define 

g^{s,y,z) = g{s,y,z) + ipi{s), {s,y,z) G [0,r] x M x W^, 

where ipi £ n'^{0,T;R) for i = 1,2. Let be the solution to BSDE lO) for 

the data {^,g) = o'^c? for the data (^,5) = (^2)52)- Then we have the 

following estimate 

t-T 



< E 



where /3 = 16(1 + C2). 

For the proof the reader is referred to El Karoui, Peng, and Quenez [9] and Peng |18j . 

Let {At,t > 0} be a continuous increasing F-progressively measurable scalar pro- 
cess, satisfying ^0 = and E[e^^'^] < 00 for all > 0. We are given a final condition 
^ G L'^{n,TT,P) such that £;(e''^T < 00 for ah /i > 0, and two random fields 
/ : 17 X [0, T] X M X M'^ ^ M and 5 : J] X [0, T] X M ^ R satisfying, 

(H2.1) (i) The processes f{-,y,z) and g{-,y) are progressively measurable and 

E[[ e^'^'\f{t,0,0)\^dt] + E[[ e''^*|5(t,0)pd.4t] < 00 for all /i > 0; 
JO Jo 

(ii) there is a constant C such that for all (t, y, z) G [0, T] x M x M'^, 

y, z) - fit, y', z')\ < C{\y - y'\ + \z- z'\); 

(iii) there exists a constant C such that for all (t, y) G [0, T] x M, 

\g{t,y)-g{t,y')\<C\y-y'\. 
A solution to the following GBSDE 

Yt = i+ I f{s,Ys,Zs)ds+ f g{s,Ys)dAs- f Z^dB^, 0<t<T, (2.2) 
Jt Jt Jt 



is a pair {{Yt, Zt),Q < t < T} of progressively measurable processes taking values in 
M X M'^ which satisfies equation (2.2) and 

[■T 

E[ sup \Yt\^] + E[ \Zt\^dt] < 00, 0<t<T. (2.3) 

0<t<T Jo 

From Theorem 1.6 and Proposition 1.1 of Pardoux and Zhang [17J, we have the 
following two lemmas. 
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Lemma 2.4. Let (H2.1) he satisfied. Then GBSDE 112. gj) has a unique solution {Y, Z). 
Lemma 2.5. Under the assumption (H2.1), we have for any fi > 



E 

< CE 



sup e^^*|ytp+ / e^'^'\Yt\^ dAt + [ e''^'\Zt\'^ dt 

<t<T Jo Jo 



0<t<T 



Jo 



+ I ef^^^git 



dA, 



(2.4) 



for a positive constant C , which depends on the Lipschitz constant of f and g, fi, and 
T. 

Let two sets of data {S,,f,g,A) and , f , g' , A') satisfy assumption (H2.1). Let 
(y, Z) is a solution to GBSDE ^ for data (C, /, g, A) and (Y', Z') for data (C', 5', A')- 
Define 

{Y,Z,U.9.A) = {J -Y',Z-Z',i-^,f -f\g-^,A-A^). 

The following two lemmas are borrowed from Proposition 1.2 and Theorem 1.4 of 
Pardoux and Zhang [T7j, respectively. 

Lemma 2.6. For any /i > 0, there exists a constant C such that 



E 

< CE 



sup e^'^'lrtp 

0<t<T 



e^''''\ZA^dt 







eMfcT|^-|2+ , e^'V{t,Yt,Zt)\'dt 



T 



(2.5) 



+ / e'^^*|5(t,yt)rdA;+ / e^'=*|5(t,yt)|^dp||i 



|2 J 4/ 







where kt := \ \A\\t + A[, and \ \A\\t is the total variation of the process A on the interval 
[0,t]. 

For the particular case A = A' , we have 

Lemma 2.7. (Comparison Theorem) Assume that ^ < f{t,y,z) < f'(t,y,z), and 
g{t,y) < g'{t,y), for all {y,z) G R x R'', dP x dt, a.s. Then Yt <Yl, 0<t< T, a.s. 
Moreover, ifYo = Y^, then Yt = Y/, < t < T, a.s. In particular, if in addition either 
< ^') > or f{t, y, z) < f'{t, y, z) for any {y, z) G R x R*^ holds on a set of positive 
dt X dP measure, or g{t, y) < g'{t, y) for any ?/ G R holds on a set of positive dA^ x dP 
measure, then Yq < Yq. 



3 Formulation of the problem and related DPP 

Let D be an open connected bounded convex subset of R"^, which is such that for a 
function G C^iR"^), D = {(p > 0}, OD = {(p = 0}, and |V^(x)| = l,x G dD. Note 
that at any boundary point x G dD, V(/)(x) is a unit normal vector to the boundary, 
pointing towards the interior of D. Let U be a metric space. An admissible control 
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process is a [/-valued F-progressively measurable process. The set of all admissible 
control processes is denoted by U. 

For an admissible control u{-) G lA, the corresponding state process starting from 
Q £ P; D) at the initial time t, is governed by the following reflected SDE: 



X 



K. 



C + J' b{r,X^/''',Ur)dr + J\{r, X^'^''' ,Ur)dBr 

iC*''^'" is increasing. 



(3.1) 



Here, we have made the following assumption on the drift 6 : [0, T] x M'' x [/ ^ M'^ and 
the diffusion a : [0, T] x M'^ x [/ ^ M'^^'^: 

(H3.1) (i) For every fixed x € M", u £ U, b{.,x,u) and a{.,x,u) are continuous in t; 

(ii) there exists a C > such that, for all t € [0,T], x,x' € M", u (zU, 
\b{t, X, u) — b{t, x', u)\ + \cr{t, x, u) — a{t, x', u)\ < C\x — x'\] 

(iii) there is some C > such that, for alH € [0, T], u G C/ and x G M", 
\b{t,x,u)\ + \a{t,x,u)\ < C(l + 

Therefore, in view of Proposition 5.1 in the appendix, SDE (j3.ip has a unique strong 
solution (X*'^'", ir*'^''") for any n(-) G Z//. Moreover, for any t G [0,T], n(-) G U, and 
C,C' € L'^in,Tt,P;D), we have 



sup 

sG[t,T] 



/|4 



sup 

se[t,T] 



< cic-C 



< c(i + !cn. 



(3.2) 



Here, the constant C depends only on the Lipschitz and the linear growth constants of 
b and a with respect to x. 

Assume that three functions 



$ : 



I, f:[0,T] X 



X K X 



U 



g : [0, T] X M'^ X 



satisfy the following conditions: 

(H3.2) (i) For every fixed {x,y,z,u) G M"^ x M x M'^ x U, f{.,x,y, z,u) is continuous in 
t; g{-) G C^'2'2([0,T] x M'^ x M); and there exists a constant C > such that, 
for aU t G [0,r], x,x' G M", y,y' G M, z,z' G M"^, n G U, 

\f{t,x,y,z,u) - f{t,x',y',z',u)\ + \g{t,x,y) - g{t,x',y')\ 
< C(|x - x'l + \y- y'\ + \z- z'\); 

(ii) there is a constant C > such that, for all x,x' G M", 

|<i>(x) - <I>(x')| < C\x - x'\. 

(iii) there exists some C > such that, for all < t < T, u £ U, and x G M", 

\fit,x,0,0,u)\<C{l + \x\). 
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Then, obviously, g and $ also have the global linear growth condition in x: there exists 
some C > such that, for all < t < T, and x € M", 

\g{t,x,0)\ + Mx)\<C{l + \x\). 

For any «(•) G U, and C S L'^{Q, Tt, P; D), the mappings ^ := <I>(Xy'''"), g{s,y) := 
g(s, X*'^'", y) and f{s,y,z) := f{s,xt''^'^,y,z,Us) satisfy the conditions i7(2.1) on the 
interval [t,T]. Therefore, there is a unique solution to the following GBSDE: 

+g{s, X'/'\Y,'''^n dK'/-^^ - Z'/'^ dB„ (3.3) 

where (x*'^'", K*'^'") solves the reflected SDE ([ST^) . 

Moreover, similar to Proposition 5.2, there exists some constant C > such that, 
for aU < t < T, C,C' e L'^{n,Tt, P; D), n(-) € Z^, P-a.s., 



(i) |y/'^^"-y/'^'"i<c[|c-c'l + lc-c'l^]; 

(ii) |y/'^^"|<c(i + |ci). 

We now introduce the following sets of admissible controls. 



(3.4) 



Definition 3.1. An admissible control process u = {ur,r G [t,s]} on [t,s] (with s £ 
{t,T]) is an Tr -progressively measurable process taking values in U. The set of all 
admissible controls on is denoted by Ut^s- We identify two processes u and u in 
lAt^s CLnd write u = u on [t,s], if P{u = u a.e. in [t,s]} = 1. 

For any n(-) G l^t,T, the value of the associated cost functional is given by 

J{t,x;u) := y/'"''", {t,x) G [0, T] x D, (3.5) 



where the process Y*'^'" is defined by GBSDE dOjl . 
From Theorem 15. H we have 

J(t,C;tx) =y/'^'", {t,C)e[0,T]xL\n,Tt,P;D). (3.6) 

We define the value function of our stochastic control problem as follows: 

W{t, x) := esssup J(t, x; u), (t, x) G [0, T] x D. (3.7) 

Under assumptions (H3.1) and (H3.2), the value function W is well-defined on 
[0, T] X D, and its values at time t are bounded and J-^-measurable random variables. 
In fact, they are all deterministic. We have 

Proposition 3.1. For any {t,x) G [0, T] x D, we have W{t,x) = E[W{t,x)], P-a.s. 
Let W{t,x) equal to its deterministic version E\W{t,x)]. Then : [0, T] x Z) — > M is 
a deterministic function. 
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Proof. The proof is an adaptation of relevant arguments of Buckdahn and Li [1]. 
Let H be the Cameron-Martin space of all absolutely continuous elements h Q whose 
derivative h is in L'^{[0,T],R'^). 

For any /i £ we define ThUJ := uj + h, a; € fi. Obviously, r/^ : — ?> is a bijection 
with the inverse r^^. The law is given by 

P o [r^-i] = exp{ / h.dBs - \ f \hs\^ds]P. 
Jo ^ Jo 

Fix any {t,x) € [0,r] x D, and define Ht := {h G H\h{-) = h{- A t)}. The rest of the 
proof is divided into the following three steps: 

Step 1. For any u G Ut^T and h G Ht, J{t,x;u) (rh) = J{t,x;u{Th)), P-a.s. 

Indeed, the r/j-shifted reflected SDE (|3.ip (with ( = x) is the same reflected 
SDE (|3.1|) with u being substituted into the r/^-shifted control process u{Th). From 
the uniqueness of the solution of the reflected SDE (j3.ip . we get Xs'^'"(r;i) = x*'^'"'^^''^ 
and Kl''^'^{Th) = i^*'^'"^'^'') for s G [t,T] P-a.s. Furthermore, by a similar shift argu- 
ment and the associated Girsanov transformation, we get from the uniqueness of the 
solution of GBSDE ([33]) that 

y/.^;«(r;,) = for any s G [t,T], P-a.s., 

Z*'^'"(r,,) = dsdP-a.e. on [t,T] x n. 

It means 

J{t,x;u){Th) = J{t,x;u{Th)), P-a.s. 
Step 2. For all h Ht we have 

{esssup J(t, x; n)}(r;i) = esssup{J{t, x;u){t}i)}, P-a.s. 
Indeed, define 

W{t, x) := esssup J(t, x; u). 

we have W{t,x) > J{t,x;u), and thus W{t,x){Th) > J{t,x]u){Th), P-a.s., for all u G 
Ut^T- On the other hand, for any random variable C satisfying (" > J{t,x;u){Th), and 
hence also C(''"-h) > J{t,x;u), P-a.s., for all u G Z//t,T) we have C{T-h) > ^(^^a^)) 
P-a.s., i.e., C ^ ^^(i, P-a-s. Consequently, 

W{t, x){Th) = esssup{ J(t, x; n)(T/j)}, P-a.s. 

Step 3. W{t,x) is invariant with respect to the shift Th, i.e., 

W{t,x){Th) = W{t,x), P-a.s., for any h e H. 

Indeed, from Step 1 to Step 2, we have, for any h G Ht, 

W{t,x){Th) = esssup^(=u^^{ J {t,x;u){Th)} 
= esssup^gi^^^ J{t,x;u{Th)) 
= esssup^gj^^ ,^ J(t,x;n) 
= W{t,x), P-a.s., 
9 



where we have used {u{Th)\u{-) G Ut^r} = Ut,T so as to obtain the 3rd equahty. There- 
fore, W{t,x){Th) = W{t,x), P-a.s. for any h G Ht. Since W{t,x) is .Ft-measurable, it 
holds for ah h £ H. Indeed, since 

Q = Co([0,T];M'^), by the definition of the filtration, 
the J^t-measurable random variable W{t,x){uj), oj € VL, only depends on the restriction 
of OJ to the time interval [0, t\. 

The result of Step 3, combined with the following Lemma |3 . 1 1 completes the proof. 



The following is borrowed from Buckdahn and Li [5, Lemma 3.4]. 

Lemma 3.1. Let C, he a random variable defined on the Wiener space {Q,J-t,P) such 
that Ci^h) = C P-O..S. for any h ^ H. Then C, = P-a.s. 

As an immediate result of (j3.4p and (|3.7p . the value function W has the following 
property . 



Lemma 3.2. There exists a constant C > such that, for all {t, x, x') € [0,T]x D x D, 

(3.8) 



(i) \W{t,x) -W{t,x')\ <C[\x-x'\ + \x-x'\2]; 

(ii) < + 



We now study the (generalized) DPP for our stochastic control problem p.ip , (j3.3p , 
and (|3.7p . For this we have to define the family of (backward) semigroups related with 
GBSDE (j3.3p . Peng |18j first introduced the notion of backward stochastic semigroups 
to study the DPP for the optimal stochastic control of SDEs. In what follows, it is 
extended to the optimal stochastic control of reflected SDEs. 

Given the initial data (t, x), a positive number 5 < T — t, an admissible control 
u(-) € and a random variable rj G L^(J7, P; M), we define 



G 



t,x;u 
■,t+S 



[ri]:=Y,''-'\ se[t,t + 6], 



(3.9) 



where (Ys'^'^, Zl'^'^)t<s<t+5 is the solution of the following GBSDE on the time interval 

rt,x\u -r^t,x:u P^t,x\u \ 7 , / -trt^x-.u -C^t,x;u\ rT^t,x:u 



[t,t + 6]: 



-dYs 



t,x:u 



Y 



t,x:u 



f{s,Xs' ' ,Ys' ' ,Zs' ' ,Us)ds + g{s,Xs' ,Ys' )dKs 
-Z'-f'^'dBs, se[t,t + 5]; 



■t+5 

and (x*'^'",i^*'^'«) is the solution of reflected SDE (f3T 



solution (y«:^;«,^t>^;«) of GBSDE ([33]), we have 



^t,x;uizr/^t,x;u\i ^t,x;u T-i^t,x;ui 
'^t,T l*(^T )\ - '^t,t+5i^t+5 \- 



Then, obviously, for the 
(3.10) 



Furthermore, 



J(t, X] u) 



G'^jUj{t+5,x\::r,u)]. 



t,x:u r-i^t,x;nn 



Remark 3.1. // both f and g do not depend on {y,z), we have 



t+s 



/(r,X^'";",^/,)dr + 



t+5 



g[r,Xj.' )dK^' 



s G [t,t+5]. 
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Theorem 3.1. Under assumptions (H3.1) and (H3.2), the value function W satisfies 
the following DPP: for any 0<t<t + 5<T, x (z D, 

W{t,x) = esssup Gl%^s[W{t + 6,xl'^n]. (3.11) 

u£Ut,t+S 

To simplify our exposition, define 

Is{t,x,u) ■.= Gl';^^,[W{t + 6,xl'^P] 

and 

Ws{t,x) := esssup Is{t,x,u) = esssup Gl'^^^[W{t + 6, xj'^^^)]. 
Similar to the proof of Proposition 13.11 we have 

Lemma 3.3. Ws{t,x) is deterministic for any 0<t<t + 6<T, x ^ D. 

The proof of Theorem 13.11 is reduced to the following two lemmas. 
Lemma 3.4. Wsit, x) < W{t, x), <t <t + 5 <T, x e D. 
Proof. For ni(-) € and U2{-) € Ut+s,T, we define 

ni e n2 := Uil[t^t+s] + ^2l(t+5,T]! 
which lies in Ut^r- Note that there exists a sequence {uj, i > 1} C Ut^t+& such that 
Ws{t,x)= esssup /^(t, X, ui) = sup /^(t, X, nj), P-a.s. 

For any e > 0, we define 

fi := {Wsit, x) < Is{t, X, u}) + e}£Tt, i> 1. 
Then the following mutually disjoint events 

Fi := Fi, Fi := rA(uj-|rO € /"t, i > 2 
form a (il, J-t)-partition. It is obvious that 



Moreover, from the uniqueness of the solution of the forward-backward SDE (FBSDE), 
we have 

Is{t,x,ul) = "^Ir J5{t,x,ul), P-a.s. 



Hence, 



Ws{t,x) < ^lrMt,x,u})+e = Is{t,x,ul) + e (3.12) 



[W{t + 5, xXf^)] + e, P-a.s. 
11 



On the other hand, from the definition of W{t + 5, y) we have, for any y £ D, 
W{t + 6,y)= esssup J{t + 6,y;u2), P-a.s. 

Finally, since there exists a constant C G M such that for any y, y' G D, 

(i) \W{t + 5,y)-W{t + 6,y')\<c(^\y-y'\ + \y-y'\iy, 

(ii) \J{t + 6,y,U2)-J{t + 6,y',U2)\<c(\y-y'\ + \y-y'\-2) P-a.s., (3.13) 

for any U2 G Ut+s^T, 

(see Lemma l3.2f i) and (j3.4p (i)) we can prove by approximating xf^^^^ that 
W{t + 6, < esssup J{t + 6, xf^f^ ; U2), P-a.s. 

To estimate the right side of the above inequality we notice that there exists some 
sequence {up j > 1} C Ut+s^r such that 

esssup J{t + 6, ',U2) = sup J{t + 6, X*^^"^ ; uj), P-a.s. 

Then, putting := {esssup^^g^^^^^^ J{t + 5, xl'^f^ ; U2) < J{t + 5, xl'^p ; uj) + e} e 
Tt+5, j > 1; we have with Ai := Ai, Aj := Aj\{ujzl^i) S ^t+s, j > 2, an {^},Tt+s)- 
partition and := J2j>i ^^j'^'j ^ ^t+5,T. Therefore, from the uniqueness of the solu- 
tion of our reflected SDE and GBSDE, we have 



J{t + 5,xXf,u\) = y;;;-''-^ (seeM) 



Thus, 



^ 1a, J{t + 6, Xf^p -u]), P-a.s. 



W{t + 5, Xl'^f^ ) < esssup„2gWt+,.T «^(* + ^> ^t*;T' ; ^2) 



i>i,t (3.14) 

= +e, P-a.s., 

where := uf © G ^t,T- Prom (13.12P and (I3.14P and Lemmas 2.7 (comparison 
theorem for GBSDEs) and 2.6, we get 

Ws{t,x) < G;;g[yi7%e]+e 

= Gl'^,^1[Ylfr'] + {C + l)e (3.15) 

= y/'-;«^ + (C + l)e 

< esssup y/'^'" + {C + l)s, P-a.s. 
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That is, 

Ws{t,x) <W{t,x) + {C + l)e. 
Finally, letting e | 0, we get Ws{t,x) < W{t,x). 

Lemma 3.5. W{t, x) < Ws{t, x), <t <t + 5 <T, x e D. 
Proof. Since 

Ws{t,x)= esssup Is{t,x,ui), 



(3.16) 



we have 



Wsit,x) > Is{t,x,ui) 

= G^:";"/ [W{t + 6, Xl'X)h ^-a.s., for all m G Ut,t+5. 

Moreover, from the definition of W{t + 5, y), y £ D, we get 

W{t + 5,y)= esssup J{t + 5,y;u2), P-a.s. 

U2&Jt+S.T 



(3.17) 



(3.18) 



Let {Oi}i>i C B{R ) be a decomposition of D such that X]j>i Oi = D and diam(Oi) 
< e, i > 1. Let yi be an arbitrarily given element of Oi, i > 1. Define 



fx 



t,x;ui 1 
t+5 i 



We have 



l^t+s^^ — < e, everywhere on Q, for all ui G 



(3.19) 



Let ti € Wt,T be arbitrarily given and decomposed into ui = njj^^^^^] € Ut^t+S and 
""2 = ^|(t+5,T] S Ut+5,T- Then, from ()3.17p . ()3.13p (i). (j3.19p . and Lemmas 2.7 and 2.6, 
we have 



W5(t,x) > G'^J^^[W{t + 5,x];X)\ 

> Gl%-i [W{t + 5, [Xl'X]) -Ce- Ce-2] - e 

Ce-C'ei P-a.s. 



(3.20) 



El 



Furthermore, from ()3.18p . (|3.13p fii). (j3.19p . and Lemmas 2.7 and 2.6, 



Ws{t,x) > Gl 



t,x;ui 
t+5 



i>l 



Ce - C'e2 



(3.21) 



Gi^;i:i[j{t+6,[xi':, 



Ce - C"e2 



> Gl'^^^l [J{t + 6, ; U2) - C"e - C'e'^] - Ce - C'e'2 



4+5 



> [J{t + 5, ; n^)] - - C'ei 

_ Y^'^''^ — Ce — C'e2, P-a.s., for any u G Z^/i.r, 

13 



where the constants C, C", C" may vary from hnes to lines. Consequently, 

(3.22) 



Ws{t,x) > esssup J {t,x;u) — Ce — C'e^ 
= W{t,x)-Ce-C'e^, P-a.s. 



Finally, letting e J, we get Ws{t,x) > W{t,x). The proof is complete. 
Remark 3.2. (i) For any u G Ut^t+&i 

W{t,x){=Ws{t,x))>G'^J:;^,[W{t + 5,xl^^--^)l P-a.s. (3.23) 

(ii) From the inequality BA^I . for all {t, x) G [0, T] x M", 5 e (0, T - t] and e > 0, 
the following holds: there exists some li^(-) G l^t,t+s such that 

W{t,x){= Ws{t,x)) < G\'^^^;[W{t + 6,Xl'^f)] + Ce, P-a.s. (3.24) 

(iii) Recall that the value function W is deterministic. Then, with 6 = T — t and 
taking the expectation on both sides of \3. 23\) and |g. 24^ we can get that 

W{t,x) = sup E[J{t,x;u)]. 

Lemma 13.21 shows that the value function W{t,x) is continuous in x, uniformly in 
t. From Theorem 13.11 we can get the continuity of W{t,x) in t. 

Theorem 3.2. Let assumptions (113.1) and (H3. 2) be satisfied. Then the value function 
W{t,x) is continuous in t. 

Proof Let {t,x) G [0, T] x D and 6 £ {0,T - t]. We want to prove that W is 
continuous in t. For this we notice that from (j3.24p . for an arbitrarily small e > 0, 

1} + Is < Wit, x) - W{t + 6,x)< 1} + Ij + C7e, (3.25) 

where 



1} ■■= Gl'^,:s[W{t + S,Xl^-^)]-Gl'^,^-,mt + 6,x)], 
Is ■■= Gl%^s[W{t + S,x)]-W{t + 6,x) 

for € Ut^t+S such that (|3.24p holds. From Lemma 2.6 and the estimate (|3.8p we get 
that, for some constant G which does not depend on the controls , 



< [CE{\W{t + 6,Xl^^-f )-W{t + 6,x)\'\Tt)]-2 

< [GE{\Xlff -x\' + \Xlff - xm)]K 

and since E[\X^'^f' — x\'^\Tt] < G5 (refer to (5.15) in the appendix) we get that 
\Is \ < C5^. From the definition of (see 



Is = E 



rt+S 

W{t + 5,x) + /(s,X*'^'"',f;*'^'"',Z*'^'"',u^)ds 



t+s ft+S 

W{t + 5,x) 



ft+S pt+S 



E 



rt+o i-t+d 

/ /(s,x*'^'^',i;*'^'^',z*'^'^',Ods+ / (7(s,x*'^■"^^;*'^'«')dE:*'^;"V^ 
Jt Jt 
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From the Schwartz inequahty, Propositions 5.2 and 5.3 in the Appendix and (13. 2p . we 
then get 



nil < 6^2 E 



+E 



< 6^^E 



-E 



t+s 



x*'-^"\ j-f 



E 



t+5 



t+5 



X*'-^- , 0, 0, <) I + C\YP^-^^^ I + CIZ*'-^- \fds\Ft 



\g{s, X*'^'"', 



t+5 



< C5^E 



+CE 



t+& 



(11 + + |y,*'^'"'|)2(ii^*'^'^Vt 



< C62 +C i^E 
Then, from (f3:25|) . 

I x) - + 5, x) I < C7(53 + + Ce, 
and letting e | we get W{t, x) is continuous in t. The proof is complete. 



4 Viscosity solutions of related HJB equations 



We consider the following PDE: 
d 



W{t, x) + H{t, X, W, DW, D^W) = 0, 



— W{t,x)+ g{t,x,W{t,x)) 



W{T,x) = ^{x), 
where at a point x G dD 



{t,x) e [o,r) X D, 

0<t<T, x£dD; 

x e D, 



(4.1) 



d_ 

' dn 



E 



a 

«=1 dxi 



(x)-^, and the Hamiltonian 



H{t, X, y,p, X) = sup < -ti{aa {t, x, u)X) + p ■ b{t, x, u) + /(t, x, y,p(T, n) 

where t € [0,r], x € M*", y G M, p S M'^,and X G S'' (recah that S'' denotes the 
set of d X d symmetric matrices). Here the two pairs of functions (6, a) and (/, 'I') are 
supposed to satisfy assumptions (H3.1) and (H3.2), respectively. 

In this section we shall prove that the value function W defined by (j3.7p is a viscosity 
solution of (j4.ip . The interested reader is referred to Crandall, Ishii, and Lions [5j for 
a detailed introduction to viscosity solutions . Let Cfj,([0,T] x D) be the set of the 
real-valued functions that are continuously differentiable up to the third order and 
whose derivatives of order from 1 to 3 are bounded. 
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Definition 4.1. A real-valued continuous function W € C([0,T] x D) is called 

(i) a viscosity subsolution of (j4.ip if W{T,x) < for all x & D and if for all 
functions ip £ Cij^{[0,T] x D) and {t,x) £ [0,T) x D such that W — (p attains its local 
maximum at {t,x): 

^(t,x) + H(t,x,W,D^,D'^^p) >0, ifxeD; 
ot 

max{^(t,x) + H(t,x,W,D^p,D^ip), ^(t,x) + g(t,x,W)\ >0, if x € dD; 
[ ot on J 

(ii) a viscosity supersolution of (j4.ip if W{T,x) > ^{x) for all x ^ D and if for all 
functions 99 G Cf^{^,T] x D) and {t,x) G [0,T) x D such that W — ip attains its local 
minimum at {t,x): 

^{t,x) + H{t,x,W,Dip,D^ip) <0, if xeD; 



at 
min 



[^{t,x) + H{t,x,W,Dip,D^ip), ^{t,x) + git,x,W)} <0, if x e OD; 



(iii) a viscosity solution of ()4.ip if it is both a viscosity sub- and a supersolution of 

For simplicity of notation, define for € Cff^{[0,T] x D), 

d 1 

F(s,x,y,z,u) = -—(p(s,x)-\ — triaa'^ (s,x,u)D'^(p) -\- Dip.b(s,x,u) (4.2) 
OS 2 

+/(s, x,y + ip{s, x), z + Dip{s, x).a{s, x, u),u), 
d 

G{s,x,y) = -^ip{s,x) + g{s,x,y + ip{s,x)) 
for (s, x, y, z, n) € [0, T] X Z) X M X E"^ X U. 

Proposition 4.1. Under the assumptions (H3.1) and (H3.2) the value function W is 
a viscosity subsolution of (j4.ip . 

Proof. Obviously, W{T,x) = $(2;), xeD. Suppose that ^ G Cf^,{[Q,T] x D) and 
G [0,r) X Z) is such that W — attains its maximum at {t,x). Without loss of 
generality, assume that (p{t,x) = W{t,x). 

We first consider the case x & D. We shall prove that 

supF{t,x,0,0,u) > 0. 

If this is not true, then there exists some 9 > such that 

Fo{t, x) := sup F{t, X, 0, 0, u) < -9 < 0. (4.3) 

Therefore, 

F{t, X, 0, 0, u) < -9 for all u £ U. 
Since Fq is continuous at (t,x), we can choose a G (0,T — t] such that 

0^{x):={y:\y-x\<a}cD, (4.4) 
F(s, y, 0, 0, u) < -\9 for all (s, y, u) G [t, t + a] x O^^ix) x [/. (4.5) 
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For any a € (0, a], consider the following BSDE: 

-Zl'^'dBs, se[t,t + a]; (4.6) 

where the pair of processes (X*'^'", ET*'^'") are given by p.ip and n(-) S Ut^t+a- It is not 
hard to check that F(s, Xg'^'", y, z, n^) and G(s, Xs'^'", y) satisfy (H2.1). Thus, due to 
Lemma 2.4, GBSDE (j4.6p has a unique solution. We have the following observation. 



Lemma 4.1. For every s € + a], we have the following relationship: 

n''" = + «, Xlf:)] - ^{s, Xl'^n, P-a.s. (4.7) 

Proof. We recall that G*J^!^^[ip{t + a, X*^^")] is defined by the solution of the 
GBSDE 

' -dY- = fis,Xl'^'^,Y,-,Z^,u,)ds + g{s,Xl'^'\Y^^)dKl'^'^ 
-Z^dBs, s G [t,t + a]; 
Y,l^ = v^(t + a,X*;^^), 

with the following formula: 

G'j:tU^{t + a, X*;T)] = n", sG[t,t + a], (4.8) 
(see (|3.9I) ). Hence, we only need to show that Y^ — ip{s, xl'^'^) = Ys'^ for s G [t,t + a]. 



This can be verified directly by applying Ito's formula to ip{s,Xs' ' ). Indeed, the 
stochastic differentials of Y^ — ip{s, xl'^'^) and Yg'^ equal, and with the same terminal 
condition Y,l^ - ^{t + a, X*;^^) = = Y^f^. 

Remark 4.1. For x £ dD Lemma 4-1 still holds. 

On the other hand, from the DPP (see Theorem 13. ip . for every a, 
^{t,x) = W{t,x) = esssup Gj;?^4",[W^(t + a,X*;"^)], 

and from W < ip and the monotonicity property of (see Lemma 2.7) we get 

esssup {Glf^^g[(p{t + a, Xj:^^)] - ip{t, x)}>0, P-a.s. 

U&At.t + a 

Thus, from Lemma l4.H we have 

esssup y/'" > 0, P-a.s. 

Hence, given an arbitrary e > we can choose G Ut^t+a such that, P-a.s., 

y/'"' > -ea. (4.9) 

(The proof is similar to that of inequality (I3.24p .) 
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Remark 4.2. Similarly, for x G dD we also have 
For € Ut^t+a define 

r = inf {s > t : -x\>a}^{t + a). 

Consequently, on [t, r] the process (/f*'^'") is zero and, hence 

J s J s 

Consider the following two BSDEs: 

1 = 

whose unique solution is given by 



2C* 



a 



and 



y3 ^ vi;n^_ 



(4.10) 



(4.11) 



(4.12) 



Here, C* is the Lipschitz constant of F with respect to y, z; also the Lipschitz constant 
of G with respect to y, in order to be different from the constant C which may vary 
from lines to lines. We have the following lemma. 

Lemma 4.2. We have y/'"' < y/ and \Y^^ - y/| < Ca^,P-a.s. Here C > is 
independent of both the control u and a. 

Proof. (1) We observe from (j4.5p and the definition of r that, for all (s, y, z, u) G 

[t, r] X M X M*^ X ?7, 

F(s,X*'"'"^y,z,n^) < C*(|y| + |z|)+F(s,X*•"'"^0,0,n^) 
< C*i\y\ + \z\)-^e. 

Consequently, from Lemma 2.2 (the comparison result for BSDEs) we have that 

Ys'^''' <Y,^se[t,T], P-a.s., 

where Y^ is defined by BSDE (imi) . 

(2) From the equation (14. 6p . Proposition 5.1 and Proposition 5.2 in the Appendix, 
we have 



y,''"' \<Cit + a-T)^ + C[E {Kl'J^f - e:*.^;«')2 I 



where C is independent of controls, and KA'^ — Kr ' = K^ 



the uniqueness of solution of reflected SDE (j3.ip . Therefore, we have 



E 



y 



l:u' 



£|2 



< [(t + a - r) I Ji] + CE 
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by means of 



From Proposition 5.3 in the Appendix, we have 

2 



E 



Therefore, we get 



K 



E 



t+a 



<C{E [{t + a-rf I Tt]) 



<C{E[{t + a-Tf I Tt\y . 



On the other hand, we consider the following SDE: 



b{s,Xt''''''%ul)ds + ais,Xl''''''\ul)dB„ s>t; X,''^'"' = x. (4.15) 



Then we know on [t,T], P-a.s. 



For X^'^'^" we have the classical estimate 



E 



sup \X, 

t<s<t+a 



t,x:u^ 



< Ca^, P-a.s. 



Therefore, we have 



C 



P{t <t + a\Tt} < P { sup [X*'^'" -x\>a\ Tt} < ^a" 



se[i,t+Q] 



a" 



Hence, 



E 



-Ft <Ca{P{T <t + a\ Tt]Y <^ 



I . C 



a 



Furthermore, from Lemma 2.3 



(4.13) 



(4.14) 



(4.16) 



(4.17) 



< C{E[\Y,^\^\Tt]y +C{E[\Y,^\^\Tt]y^ 

< Cf (1 - e-^*") (P{T < t + a| J-J)^ + C (E[\Yr'-''''\^\Tt]) ~' (4-18) 

< C|(l-e- 

< Cal, 



for any a G (0, a]. 

The above auxiliary results now allow to complete the proof of Proposition 4.1. 

Proof of Proposition 4.1 (sequel). 

By combining (j4.9|) with Lemma 4.2 we then obtain 

-ea < y/'"' < < + jy^^ - y/| < y*^ + Cai, p-a.s. 

i.e., 
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2C* 
Therefore, 

9 1 

-e < --^ + Ca2. 

2C* a 

Letting a — t- 0+ and e 0+, we get < — |, which contradicts our assumption that 
6 > 0. Therefore, we have 

supF{t,x,0,0,u) > 0, 
which impHes by the definition of F that 

^(t,x) + H(t,x,W,Dip,D'^^p) >0, iixeD. 
ot 



We now consider the case x G dD. We must prove that 

r dip d 
max <^ — (t, x) + H{t, x, tp, Dip, D^ip), —{t, x) + g{t, x,ip)} >0 

If this is not true, then there exists some 9 > such that 



sup F{t, X, 0, 0, u) <-e < 0, G{t, x, 0) <-e < 0, (4.19) 

therefore, 

F{t, X, 0, 0, u) < -9 for all u £ U. 
G{t, X, 0) < -9 for ah u e U. 
Choose a G (0, T — t] such that 

F{s,y,0,0,u) <-^9, (4.20) 
G(s, y, 0) < -^9, for all u eU,t < s <t + a, \y - x\ < a. (4.21) 

Now we fix a, and we consider any a € (0, a]. Similarly, we consider GBSDE (4.6) 
with X € dD, then we also can get (4.7) and (4.9). For u'^ S Ut^t+a in (|4.9p define 

r = inf {s > t : - j;| > a} A (t + a); 

We observe that, for all {s, y, z) G [t, r] x M x M'^, from (g^OD, (li:2T]) and the definition 

F{s,X'r'''\y,z,ul) < C*{\y\ + \z\)+F{s,X'r^''\0,0,ul) 
< C*i\y\ + \z\)-l9. 

G{s,Xt'^'''\y) < C1y| + G(s,X*'^^"^0) 
< C*\y\-^9. 

Consequently, applying the comparison result for GBSDEs (Lemma 2.7, or Remark 1.5 
in Pardoux and Zhang [T7]) to GBSDEs (4.6) and (4.23) we have that 

Y}^-' <Y,\s€[t,r], P-a.s., (4.22) 
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where is defined by the following BSDE: 

On the other hand, we also have to introduce the following BSDE: 

/ -dY,' = + |Zf I) - '^9}ds + (C*|y/| - ^9)dK'r'''' - Z^dB,., 



Notice that C*\Y^\ - ^9 < 0, therefore Y^ < Y^,s G [t,t + a], P-a.s., from the 
comparison theorem-Lemma 2.7. 
From Lemma 2.6 we have 



ly^-y*^! < c{E[\Y,^-Y^\'\Tt]) 



< C{E[\Y,r\^t]y^+C{E[\Y,'\'\Tt]y 

1 ^ 

2 



< C [E[\Yr'''''\^\Tt]) ' + C {E[\Y:^\''\Tt]Y + C {E[\Y^ - Y^\^\Ft]y- 

< Cal + C {E[\Y^^ - y,2[2|j-^])5 (from the proof of glS}), 

for any a G (0, a]. From (5.16) of Remark 5.3 in the Appendix, similarly we also have 



P{t <t + a\J^t} < P < sup -x| > a I ^ < ^a^. (4.25) 

On the other hand, from Lemma 2.6 (taking /i = 1) 

E[\Y^-Y,'\^\Tt] 



< C^{P{t < t + a\J't})HE[{t + a - Tf\J't])i (from Prop. 5.1 and 5.3.) 



^■{P{T < t + a\J^t}r^{E[{t + a - T)-| J-tJJ ' 

< C9^iP{T <t + a\Tt})^a'^P{T < t + a|j;})3 

< C9^al. 

Therefore, 

ly*^ - ^'t^l < Cal + C70a3. (4.27) 

Now we obtain 

-ea < y/'"' < y/ < y/ + |y/ - y/| < y^^ + Cai + C^at, p-a.s. 

i.e.. 



-ea < y/ < -— ( 1 - e"^ +Ca2 +C9a^, P-a.s. 



'2C7 
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3 5 



Therefore, 



-e < - — + Ca2 + Cea-i , 

26* a 



and by taking the hmit as a J, 0, e J, we get < — | which contradicts our assumption 
that > 0. Therefore, it must hold 

max + ^{t,x) + g{t,x,W) \ > 0. 



In an identical way, we can show 

Proposition 4.2. Under the assumptions (H4.1) and (H4.2), the value function W is 
a viscosity supersolution to (j4.ip . 

Proof. Obviously, W{T,x) = ^>(x), x e D. Suppose that ^p G Cj^^([0,r] x D) and 
{t,x) € [0,r) X Z) is such that W — (p attains its minimum at {t,x). Without loss of 
generality, assume that f{t,x) = W{t,x). 

We first consider the case x & D. We shall prove that 

sup x, 0, 0, -u) < 0. 

If this is not true, then there exists some 9 > such that 

Fo{t, x) := sup F{t, x, 0, 0, n) > 6* > 0. (4.28) 

Therefore, there exists a n* = u*{t,x) E U such that 

F{t,x,0,0,u*) > —. 

Since Fq is continuous at {t,x), we can choose a G (0, T — t] (for simplifying the 
notation, we still use a ) such that 

Oaix) ■.= {y:\y-x\<a} CD, (4.29) 
F{s,y,0,0,u*) > \e for all (s,y) G + a] x Os,{x). (4.30) 

For any a G (0, a], we still consider the BSDE (14. 6p : 

' -dy/'" = F(s,x*'^'",y,i'",zl'",u,)(is + G(s,x*'^'",y,i'")dK*'^'" 

-Zl'^'dBs, se[t,t + a]; (4.31) 

yl," _ n 
^t+a — 

where the pair of processes (X*'^'", i^*'^'") are given by (3.1) and u{-) G Ut^t+a- There- 
fore, Lemma 4.1 still holds for x & D. 

On the other hand, from the DPP (see Theorem 3.1), for every a, 



f{t,x) = W{t,x) = esssup Glt]^^[W{t + a, Xl'^^)], 

u£Ut,t + a 

22 



and from W > tp and the monotonicity property of (see Lemma 2.7) we have 



esssup {G^^i^g[ip{t + a, X^+a")] - ip{t, a;)} < 0, P-a.s. 

U&Ut.t + a 



Thus, from Lemma l4.H we get 

esssup y/'" < 0, P-a.s. 

UGUt,t + a 

Hence, P-a.s., 

y/'"* < 0. (4.32) 



Remark 4.3. Similarly, for x € dD we also have li4-32 ). 
For u* E Ut^t+a we define 

r = inf {s > t : - x\ > a} A {t + a). 

Consequently, on [t,T] the process (i^*'^''"*) is zero and, hence 

y/'«* = y^i^«* + rF(r,X^'^;"',y^i;"*,Zi'"*,tx*)dr- f Z}:""* dBr- 

J s J s 

Consider the following two BSDEs: 

(dY^2 ^ [-C*i\Z^\ + \Z^\)+^^e]ds-Z^dBs, 
1 Y,l^ = 

whose unique solution is given by 



and 



-dy/ = [-C*{\Z^\ + \Z^\)+'^6]ds-Z^dBs, se[t,T] 

y3 = yi;-*. 



We have the following lemma. 



(4.33) 



y,2 = ^(l-e^*(-(*+"))),Z,2 = 0, se[t,t + a], (4.34) 



(4.35) 



Lemma 4.3. We have y^ > and \Y^ - Y^\ < Ca^, P-a.s. Here C > 
independent of both the control u and a. 



IS 



Proof. (1) We observe from (I4.30p and the definition of r that, for all (s,y,z,u) G 
[t, t] xRxR'^ xU, 

F{s,X'f^''\y,z,u*) > -C*(|y| + |z|) + F(s,X*'"'"*,0,0,n*) 
> -C*{\y\ + \z\)+'^d. 

Consequently, from Lemma 2.2 (the comparison result for BSDEs) we have that 

Y^ >Z\se[t,T], 
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where is defined by BSDE (05]) . 

(2) Prom the equation (4.31), Propositions 5.1 and 5.2 



<C{t + a-T)2+C[E {Kl 



rt,x;u* _ j^t,x;u*\2 



where C is independent of controls. Then similar to the proof of estimate (j4.14p . we 
have 



E 



<C{E{{t-ra-Tf I Tt\) 



Similar to (|4.16p . we still have 



C 



a" 



Therefore, 



E 



1 (J 

< Ca {P {t < t + a I Tt}) ^ < -^a^ 



Purthermore, from Lemma 2.3 



< C(^E[\Y,^-Y,^\^\Tt 

< c(E[\Y^^\^\Tt]Y' +C(E[\t^\^\Tt] 



< C|(l-e-^"-)(P{T <t + a!7-t})t + C(i?[|y-''"*|2|J-i] 
e 

2 



< C^(l-e-^*-)^a2 + fat 



< Ca2, 
for any a G (0, a]. 

Now we complete the proof of Proposition 14. 2[ 

Proof of Proposition 14.21 (sequel). 

By combining (j4.32p with Lemma 4.3 we then obtain 

A 'a* 



> y/'" > y/ > y/ - |y/ - y/| > y/ - Caa , p-a.s. 



I.e., 



(4.36) 



(4.37) 



(4.38) 



(4.39) 



Therefore, 



> y/'"* > 



> 



2C* 



l-e 



-C*a 



Ca^, P-a.s. 



l-e 



-C'a 



Ca-2 . 



2C* a 

Letting a 0+ , we get > |, which contradicts our assumption that ^ > 0. 
Therefore, we have 

supF(t,x,0,0,u) < 0, 
ueu 
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which impUes by the definition of F that 

^'^-{t,x)+H{t,x,W,D<f,D'^ip)<{), if xeD. 



dt 

We now consider the case x G dD. We must prove that 

min <^ — (t, x) + X, ip, Dip, D'^ip), — (t, x) + g{t, x,ip) \ <0 

If this is not true, then there exists some 9 > such that 

sup F{t, X, 0,0,u)>e> 0, G{t, X, 0) > 6* > 0, (4.40) 

therefore, there exists u* such that 
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F(t,x,0,0,n*) > y. 
Choose a G (0, T — t] such that 

F(s,y,0,0,^/*) > (4.41) 
G{s, y, 0) > i(9, for all t < s < i + a, |y - x| < a. (4.42) 

Now we fix a, and we consider any a € (0,a]. Similarly, we still consider GBSDE 
(4.31) with X G dD. For this G Ut^t+a we still have (14.32j) and define 

T = inf {s > t : - x[ > a} A (t + a); 

We observe that, for all (s, y, z) G [t, r] x M x M'^, from ()4.4ip . ()4.42p and the definition 

F{s,X'r'''\y,z,ul) > -C*{\y\ + \z\) + F{s,X'r'''\0,0,ul) 
> -C*{\y\ + \z\) + l9. 

G(s,X*'"'"*,y) > -C*|y| + G(s,X*'"'"*,0) 
> -C*\y\ + l9. 

Consequently, from the comparison result for GBSDEs (Lemma 2.7, or Remark 1.5 in 
Pardoux and Zhang [T7]) we have that 

Y}'-' >Y^\se[t,T], P-a.s., 

where is defined by the following BSDE: 

r -dy/ = {-c*(|y/| + |z4|) + i^jds + (-c*iy/[ + i^)^;^*.-;"* _ z^dB,, 

(4.43) 

On the other hand, we also have to introduce the following BSDE: 



dy/ = {-c*(|y/| + \zl\) + ^9}ds + (-c*iy/| + i0)dii:*'^'"* - zfds,, 

(4.44) 
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Notice that -C*\Y^\ + ^9 > 0, therefore Y,^ > , s e [t,t + a], P-a.s., from Lemma 
2.7. 

From Lemma 2.6 we have 



\Yf-Y^\ < c(^E[\Y^^-Y^^\^\Tt] 



< C(E[\Y,^\'\Tt])+C{E[\Y;-\'\Tt 



< C(E[\Yr'''''\^\Tt]Y + C (E[\t^\^\Tt]) ' + C (E[\t^ - t^l^lTt"^ ' 



1 



(4.45) 



(4.46) 



< Cai + Ci^E[\Y^ -Y^\'^\Tt]) (from the proof of gJlD), 

for any a € (0, a]. 

Similar to (4.25) and (4.26), P{t <t + a\Tt} < ^a*^; and 

E[0-%^\^\Tt] 
< CE[X_*+"e2^»*'""*(C*|n2| _ i0)2di^*.-;«*|j-,] 

Therefore, 

\Yt^ -Y^\< Cat + COa-^. (4.47) 

Now we obtain 

> y/'"* > > - 1?;^ - y/l > y*^ - cat - c^at, p-a.s. 

i.e., 

Therefore, 

9 1- e"^*° 1 1 

> — Ca2 - C9a^ , 

- 2C* a 

and by taking the hmit as a J, 0, we get > | which contradicts our assumption that 
9 > 0. Therefore, it must hold 

mm!^^{t,x)+H{t,x,W,Dip,D^^), ^{t,x) + g{t,x,W)^ < 0. 

□ 

Therefore, we have 

Theorem 4.1. Under the assumptions (H4.1) and (H4.2), the value function W is a 
viscosity solution to (j4.ip . 

Remark 4.4. Uniqueness of viscosity solutions to elliptic equations with nonlinear 
Neumann boundary condition can be found in Barles fl^, Bourgoing f^jjl, and Crandall, 
Ishii, and Lions |3 Section 7B], etc. Uniqueness of viscosity solutions to a system of 
parabolic PDEs in the whole space can be found in Buckdahn and Li 
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5 Appendix: Forward-Backward SDES (FBSDEs) 



In this section we document some necessary basic results on GBSDEs associated 
with forward reflected SDEs (for short: FSDEs). 

We consider measurable functions b : [0, T] x $7 x M'' — > M'^ and u : [0, T] x $7 x M"^ — > 
j^dxd -^^j^ich supposed to satisfy the following conditions: 

(H5.1) (i) 6(-,0) and (t(-,0) are F — adapted processes, and there exists some 
constant C > such that 

\b{t,x)\ + \a{t,x)\ < C(l + |2;|),a.s., for ah < t < T, x£ R'^; 
(ii) b and a are Lipschitz in x, i.e., there is some constant C > such that 
\b{t, x) — b{t, x')\ + \(T{t, x) — a{t, x')\ < C\x — a.s., 

for ah < t < T, x, x' G M'^. 



Under the assumption (H5.1), it follows from the results in Lions and Sznitman |12j 
that for each initial condition (t, C,) € [0, T] x L^($7, J-t,P', D) there exists a unique pair 
of progressively measurable continuous processes {(X*'^, i^*'^)}, with values in Z) x 
such that 

= C + // b{r, x'/)dr + // a{r, x'/)dBr + // Vcl){x'/)dK'/ , s £ [t, T], 
^^s^ = // /^^t.Cgg^ldi^r^, -ftT*''' is increasing. 

(5.1) 

Proposition 5.1. For each T > 0, there exists a constant Ct such that for all Q, Q' G 

E{ sup - Xl^^'t\Tt) < Ct\C - CT, (5.2) 

t<s<T 

and 

E{snp \K'/-K'/'\'\Tt)<CTK-C'\'- (5.3) 

t<s<T 

Moreover, for each fi > 0, s G [t, T], there exists C{fi, s) such that for all ( E L^(0, Tt, P; D), 

E{e^'''^''\Ft)<C{fi,s). (5.4) 

The proof is similar to that of Propositions 3.1 and 3.2 in Pardoux and Zhang |17j . 
Assume that the three functions /(t, x, y, z),g{t, x, y) and ^{x) satisfy the following 
conditions: 

(H5.2) (i) ^> : X M'^ ^ M is an J-T ;B(M'^)-measurable random variable and 
/ : [0, T] X X M*^ X M X R'^ ^> M is a measurable process such that 
/(., X, z) is F-adapted, for ah (x, y, z) G M"^ x M x W^] 
g : [0, T] X M'^ X M — 7- M is a measurable function such that 
g{-) G Ci'2'2([0,r] xR'^ X M); 

(ii) There exists a constant C > such that 

\fit,x,y,z) - f{t,x',y',z')\ + \g{t,x,y) - g{t,x' ,y')\ + |$(x) - ^>(x')| 
< C(|x — x'l + \y — y'\ + \z — z'\), ct.s., 

for all < t < T; x, x' G M"'; y, y' G M; z, z' G M'^; 
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(iii) / and $ satisfy a linear growth condition, i.e., there exists some C > 
snch that, dt x dP-a.c., for all x <E W^, .x, 0, 0)| + |$(x)| < C(l + |x|). 

Under the above assumptions the coefficients f{s, X*''', y, z) and g{s, Xl''^ ,y) satisfy 
(H2.1) and ^ = <i>{X^/) G L^{n,J='T,P). Therefore, the following GBSDE possesses a 
unique solution: 

-dF,*'^ = f{s, X'/, Ys''^, Z'/)ds + g{s, Xl'^ X'^)dKl'^ - Z'/dB^, s G [t, T], 

(5.5) 

Proposition 5.2. Let assumptions (H5.1) and (H5.2) hold. Then, for any <t <T 

(i) E[svj£)t<s<T + !t \Zl'^?ds\Tt] < C(l + |Cp), a.s.; and in particular, 

|y/'^|<C(l + |C|), a.s- 

(ii) |f/'^-F/'^'|<CK-C'I + C|C-C'I^ a.s., 

where the constant C > depends only on the Lipschitz and the growth constants of b, 
a, f, g and 

Remark 5.1. Since D is bounded, we have 

E[sup / \Zl''^\'^ds\Tt]<C, a.s., (5.6) 

t<s<T Jt 

where C is independent of Q. 

Proof. From Lemma 2.5 and Proposition 5.1, we have assertion (i). Now we prove 
assertion (ii). First notice that from (i) we have \Y^'^\ < C(l + |C|), a.s., therefore we 
can get from the uniqueness of the solution of equations (5.1) and (5.5) that 

\yU\ = \Y,txl'<^ < ^ [^t,C|) < c, a.s., (5.7) 

since D is bounded. From Burkholder-Davis-Gundy inequality and (5.5), as well as 
from the boundedness of the processes 

E[{J^ \Z'/\^drf\Tt] 

< C^[sup,g[,,r] I Z'/dB.l^lTt] 

<C + CoiT - sfE[{jJ \Z'/\''drf\Ft] + C E[{K'^^f\J^t] 
<C + Co{T- sfE[{jJ |Z*'^|2dr)2|.F,]. 

Consequently, for T - s < (3^)^^^, E[{jJ \Zl''^\^drf\Tt] < C. This argument allows 
to choose a partition t = to < ti < ... < t]\f = Tof the interval [t,T] such that 
^[(/tli \Zr'''\'^drf\J't] <C,l<i<N. Therefore, we have 

E[{£ \Z'/fdrf\J^t]<C. (5.8) 
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For any A > 0, applying Ito's formula to e^^^'*" l^s*'^ — Yg'^ p, we have 

_ y^t.C |2 + X /J e^^r'^' |y/'^ - y/'^' \^dK'/' + /J e^^*''' - Z'/'\^dr 

+2/Je^^*'^'(y.*'^ - Yr'^^'){f{r,Xp<X'^,Z'/) - f{r,X'/'X'^',Zp<'))dr 
+2 /J e^^>'' {Yr''< - Yr''^'){gir, X'/ ,Yr''<) - g{r, X'/' X'^'))dK'/' 
+2 /J e^^'r'' iYr''< - Yr'^^')g{r, Xp< ,Yr'MK'/ - K'/') 
-2 f e^^*''' (y/'^ - y/'^') < Z*'^ - Z'/',dBr > . 

(5.9) 

Then from (H5.1), (H5.2), (5.4), (5.7) and (5.8), taking a suitable A > 0, we get: 

ly/'C-yi'f'l^ 

< C\C - C? + CE[J^ e^^*''' \Yr''^ - Y^^^'\^dr\Ts] 

+CE[J^e'^'''yp<-Xp<)'dr\Ts] + CE[f^e'^>']xp<-X^^^^^ 
+E[2 Jj e^^*''' (y/'^ - y.*'^')5(r-, X*'^ y/'^)d(i^*'^ - i^*'^')|^,]. 

(5.10) 

Furthermore, from Proposition 5.1, we have 

|yi'C_y*.C',2 

< C|C - Cf + C^[/f e^^*'^' l^r'^ - Y}^'^'\'^dr\Fs\ (5.11) 

+E[2/Je^^*'''(y/'^ - y/'^')5(r,x*'^,y/'^)d(i^*'^ - i^F)|^s]. 

On the other hand, applying Ito's formula to e^^*'*^' {Yg'^ - Yg''^')g{s, xt''^ ,Yg''^){Kt''^ - 
Kg ), we have 

E[f^ e'^r'^'iYr'^^ - Yr''<')g{r,X'/,Yr'MK'/ - i^^'^')|^.] 
= E[e^^T'' {Y^'< - Y^'^')g{T, x!^"^ ,Y^'<){K'/ - K'/ m] 

+E[f^ hir){K'/' - K'/)dr\Tg] (5-12) 
+E[jJ f2{r){K'/' - K*/)dK*/\Ts] 
+E[JJ h{r){K'/' - Kl^^)dKl^^\Fgl 

where 

= -eA^^^'(/(.,xF,y5,zF) - f{s,X^^,Y^'^,Z^^))g{s,X^^,Y^^^) 
+,AK^^'(yU _ y«'){£5(s,x*'^y.*''^) + V^g{s,Xl^^,Yt^)h{s,Xi^^) 
-Vyg{s, X*'^ y,*'«)/(s, X*'^, y,*'«, Z*'';) + itr(i52^(,s, X*''^, y,*''^)aa^(.s, X*'^)) 
+ iZ)2^(5,xF,ys'^)|Zi'^P + itr < a,yg{s,X'/,Y,'^'^)a{s,Xi^^),Z'/ >} 
+e^^*''' (Z*-'^ - Zt'^'nVMs, X'/,Ys'-^^)a{s, Xl'^) + V,5(.'^, X'/ ,Yg''^)Z's^}; 

f2{s) = {Ae^^^'''(y,*'^ - y,*'^') + e^^^'''5(s,Xi'^',n*'^')}5(5,X*'Sy,*'^); 

/3(,) = e^^"'' (y,*'^ - y.*''^'){v,<7(s, x*'«, y,*'«)V0(s, x*'^) - Wyg{s, x*'S yi'^)5(s, x*'«, y.*''^)} 

-e^^"'''b(s,X*'^yi'^)|2. 

29 



Prom assertion (i), Propositions 5.1, (5.7 )and (5.8), we have 
Furthermore, from (5.11) and (5.4) we have 

E[|yi'f-y5'|^|/-t] 

< C\C - Cr + C\C - Cf + CE[{J^ e^^*''' - Yr''^'\^drf\Tt] 
<C\C- Cr + C\C - cf + CEie^^^r' \Tt]E[f^ \Yr''^ - Yr'^^' ^J^t] 

< c\c - cr + c\C - Cf + CE[J^ |y/'^ - y.*'^rdr|^t], s e [t, r], 

then from Gronwall's Lemma, we get E[\Ys''^ - y^'^'l^'l^t] < C|C - Cf + C'lC - Cf , 
a.s, s G [t,T], which means (ii) for s = t. □ 

Remark 5.2. If g is a bounded random variable, assertion (ii) of (5.6)still holds. 
Indeed, from Lemma 2.3 and Proposition 5.1, we get 

|y^t,C_yU'|2 < CE[\C-C' + g{uj){K'i^-K'/)\^\Ft] 

+CE[jJ\f{s,Xl'^X'^,Zl'^)-f{s,X'/,Yt^,Zl^^)\^ds\Ft] 
< C|C-Cf, a.s. 

Proposition 5.3. Let Hypotheses (H5.1) and (115. 2) hold. Then, for any < a < 
T — t and the associated initial conditions C G L^{Q,,J^t,P;D), we have the following 
estimates: 



E 



\Ktij''\^t <Ca,a.s., (5.13) 



where the constant C > depends only on the Lipschitz and the growth constants of b, 
a, f, g and 

Proof. For C' G L^{Q.,J^t,P;D), from Ito's formula we have 

- Cf = IC - Cf + 2//(X*'^ - C')b{r, X'/)dT + 2/;(X*'^ - C')^(r, ^r'^)c/S, 
+ /;Kr,X*'^)pdr + 2/;(X*'^-C')V</>(X*'^)di^*'^ se[t,T]. 

(5.14) 

Since D C M*^ is convex, we have 

- C,'W<i>{X*/)dK*/ < 0. (5.15) 

Therefore, we have 

E[ sup |x*'^-c'p|7;]<c(|c-Cf + «)■ 

Recall that D is an open connected bounded convex subset. In particular, we have, 

E[ sup - CH Ji] < Ca. (5.16) 

se[t,t+a] 
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Because (j) G Cl{W^) we have 

(t^iXl'^) = </)(C) + /;V(/<(X*'^)6(r,X^'^)dr + /;V0(X*'^)c7(r,X*'^)d5, 

+i //tr(Z^2^a(r,X*''^)a^(r,X*'^))dr + // |V0(X*'^)|2dK*'S s G [t,r]. 

Therefore, we get 

i^'*'^<|<A(X*'^)-<A(C)| + C /'(l + l^^'^lV+I rVcPiX'/)a{r,X'/)dBr\, 

Jt Jt 

and furthermore, from Burkholder-Davis-Gundy inequahty, we have 
E[\Kl'<J\^t]<CE[ sup |X*'^-Cp|7-t] + Ca. 

se[t,t+a] 

In view of ()5.16p . the proof is complete. □ 

Remark 5.3. In view of (5.13) and (5.14), using Burkholder-Davis-Gundy inequality, 
we have 

E[ sup \Xl^^ -C\^\Ft]<Ca^. (5.17) 

Let us now define the random field: 

u{t,x) = yi'^|s=t, {t,x) € [0,T] X D, (5.18) 
where y*'^ is the solution of GBSDE ([53]) with x € l) at the place of C S ^^(Jl, /"t, P; Z)). 

Proposition 5.2 yields that, for all t € [0,T], P-a.s., 

(i) \u{t,x) -u{t,y)\ <C\x -y\ + C\x -y\^, iov dl\ x,y e D; 

(ii) |u(t,x)| < C(l + |x|), for allx^D. 

Theorem 5.1. Under the assumptions (H3.1) and (H3.2), for any t G [0,T] and 
C e L'^{Q,Tt,P;D), we have 

u{t,C) = y/'^ P-a.s. (5.20) 

The proof of Theorem 15. II is similar to that of Theorem 3.1 in Peng [T8] or Theorem 
A. 2 in Buckdahn and Li [1]. Therefore it is omitted here. 
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